Method and Apparatus for Surface Inflation Using Mean Curvature Constraints

ABSTRACT

Method and apparatus for the interactive enhancement of 2D art with 3D geometry. A surface inflation tool may be used to create a 3D shape by inflating the surface that interpolates the input boundaries. The surface inflation tool may, for example, obtain a closed 2D boundary as input, triangulate the area within the boundary to generate an initial surface, and inflate the surface while maintaining a fixed boundary. Using the mean curvature specified at boundary vertices as a degree of freedom, the tool may control the inflated surface efficiently using a single linear system. Embodiments handle both smooth and sharp position constraints. Position constraint vertices may also have curvature constraints specified for controlling the inflation of a local surface.

PRIORITY INFORMATION

This application claims benefit of priority of U.S. ProvisionalApplication Ser. No. 61/037,240 entitled “Method and Apparatus forModeling 3D Surfaces from 2D and 3D Curves” filed Mar. 17, 2008, thecontent of which is incorporated by reference herein in its entirety,and to U.S. Provisional Application Ser. No. 61/091,262 entitled “Methodand Apparatus for Modeling 3D Surfaces from 2D and 3D Curves” filed Aug.22, 2008, the content of which is incorporated by reference herein inits entirety.

BACKGROUND DESCRIPTION OF THE RELATED ART

In computer-aided image design, sketch based interfaces have becomepopular as a method for quick three-dimensional (3D) shape modeling.With an ever-increasing set of modeling features, the powerful 3Dsketching interface can construct shapes that range from rectilinear,industrial objects to smooth, organic shapes. Usually, the applicationof these shape sketching interfaces is 3D shape design. Thetwo-dimensional (2D) curves drawn by the user are simply a means to getto the end result: the 3D shape.

Currently, there are a few 3D shape modeling tools in readily available2D artistic design software. A commonly used modeling primitive is shapeextrusion, where a closed 2D curve is swept along a straight line (or acurve) to create prismatic shapes. Similar to extrusion is curverotation, where a curve is rotated along an axis to constructrotationally symmetrical shapes. Another commonly used primitive isbeveling, where the input 2D curve is offset and raised to provide anappearance of a 3D shape that has thickness and sharp edges. These 3Dmodeling primitives are limited in the type of surface features they cansupport. For example, no conventional image design application supportsadding sharp creases in the interior of a beveled image.

A current effort of research is to improve the range of surface editspossible in a 2D design tool. Research areas include virtual embossingand a more general, function-based surface modeling system. Both use animplicit surface representation to model their surfaces (the surface isinteractively polygonized for rendering purposes).

SUMMARY

Various embodiments of a method and apparatus for the interactiveenhancement of two-dimensional (2D) art with three-dimensional (3D)geometry are described. Embodiments of a surface inflation method, whichmay be implemented as or in a tool, module, plug-in, stand-aloneapplication, etc., may be used to create a 3D shape by inflating thesurface that interpolates the input boundaries of the surface. Forsimplicity, implementations of embodiments of the surface inflationmethod described herein will be referred to collectively as a surfaceinflation tool. By using mean curvature values specified for boundaryvertices as a degree of freedom, embodiments are able to control theinflated surface efficiently using a single linear system. Embodimentshandle both smooth and sharp position constraints. Position constraintvertices can also have curvature constraints for controlling theinflation of the local surface. Embodiments may be applied in one ormore of, but not limited to, font design, stroke design, photoenhancement and freeform 3D shape design.

Typically, the application of shape sketching interfaces is in 3D shapedesign. Conventionally, the two-dimensional (2D) boundaries drawn by theuser are simply a means to get to the end result: the 3D shape. Insteadof using the 2D boundary to design 3D shapes, embodiments use theresulting interpolated 3D shapes to enhance the existing 2D boundaries.That is, embodiments may apply the shape-sketching interface to another,highly interesting application: 2D art creation and design. In contrastto conventional methods that use an implicit surface representation tomodel surfaces, embodiments may use a polygon (triangle) mesh to inflatethe given boundary. Using the inflation metaphor, embodiments mayconstruct a 3D surface that interpolates a closed input boundary.Embodiments may allow designers to modify the inflated surface with 3Dshape modeling features such as sharp creases, smooth interpolationcurves and local curvature control. Embodiments demonstrate thatsophisticated geometric modeling techniques otherwise found in 3D shapemodeling tools can effectively be used to design interesting lookingimages.

Embodiments of the surface inflation tool construct the 2-manifoldsurface that interpolates the input boundary. The surface may becomputed as a solution to a variational problem. The surface inflationmethod may be formulated so as to solve for the final, inflated surfacein a single, sparse linear equation, without requiring an additionalstrip of triangles at the boundary. In the formulation, the meancurvature of the vertices on the boundary is a degree of freedom; thesurface may be inflated by increasing these mean curvature values. Dueto the variational setup, the inflated surface may always be smoothexcept near position constraints (internal and external boundaries). Thedesigner can add and modify internal boundaries as position constraintsat any time of the design phase, and these position constraints can besmooth or sharp. Moreover, the internal boundaries can also havecurvature control (similar to the external boundary), thereby allowingthe designer to locally inflate or deflate the surface near the internalboundary.

Embodiments of the surface inflation tool may use the inflation metaphorand allow both smooth and sharp internal boundaries as positionconstraints drawn directly on the inflated surface to modify thesurface. Embodiments may use the mean curvature constraints to controlthe amount of inflation and to formulate the problem as a single linearsystem, unlike conventional methods.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates examples of results of surface inflation using anembodiment of the surface inflation tool.

FIGS. 2 a through 2 c graphically illustrate workflow in a surfaceinflation tool according to one embodiment.

FIG. 3 is a flowchart of a surface inflation method according to oneembodiment.

FIG. 4 illustrates a mesh vertex and one-ring neighbors according to oneembodiment.

FIG. 5 illustrates a mesh vertex of position x and one-ring neighborsy_(i) according to one embodiment.

FIG. 6 is a general flowchart of a method for re-initializing andsolving the linear system, according to embodiments.

FIGS. 7 a through 7 d graphically illustrate steps for addingconstraints according to one embodiment.

FIGS. 8 a through 8 c illustrate oriented position constraints accordingto one embodiment.

FIG. 9 illustrates pixel-position constraints according to oneembodiment.

FIGS. 10 a through 10 e illustrate an example of 3D font designaccording to one embodiment.

FIGS. 11 a through 11 d illustrate an example of stroke design accordingto one embodiment.

FIGS. 12 a and 12 b show an example of photograph inflation according toone embodiment.

FIGS. 13 a through 13 h graphically illustrate freeform 3D shape designaccording to one embodiment.

FIGS. 14 a and 14 b show an example surface generated with a smoothposition constraint and with a concave angle at the boundary asspecified using surface normal constraints, according to one embodiment.

FIGS. 15 a and 15 b show an example surface generated with a smoothposition constraint and with a flat angle at the boundary as specifiedusing surface normal constraints, according to one embodiment.

FIGS. 16 a and 16 b show an example surface generated with a smoothposition constraint and with a convex angle at the boundary as specifiedusing surface normal constraints, according to one embodiment.

FIGS. 17 a through 17 c illustrate modifying the angle(s) at an internalboundary of an example surface using surface normal constraints,according to some embodiments.

FIG. 18 illustrates an example embodiment of a surface inflation tool.

FIG. 19 illustrates an example of a computer system that may be used inembodiments.

While the invention is described herein by way of example for severalembodiments and illustrative drawings, those skilled in the art willrecognize that the invention is not limited to the embodiments ordrawings described. It should be understood, that the drawings anddetailed description thereto are not intended to limit the invention tothe particular form disclosed, but on the contrary, the intention is tocover all modifications, equivalents and alternatives falling within thespirit and scope of the present invention. The headings used herein arefor organizational purposes only and are not meant to be used to limitthe scope of the description. As used throughout this application, theword “may” is used in a permissive sense (i.e., meaning having thepotential to), rather than the mandatory sense (i.e., meaning must).Similarly, the words “include”, “including”, and “includes” meanincluding, but not limited to.

DETAILED DESCRIPTION OF EMBODIMENTS

In the following detailed description, numerous specific details are setforth to provide a thorough understanding of claimed subject matter.However, it will be understood by those skilled in the art that claimedsubject matter may be practiced without these specific details. In otherinstances, methods, apparatuses or systems that would be known by one ofordinary skill have not been described in detail so as not to obscureclaimed subject matter.

Some portions of the detailed description which follow are presented interms of algorithms or symbolic representations of operations on binarydigital signals stored within a memory of a specific apparatus orspecial purpose computing device or platform. In the context of thisparticular specification, the term specific apparatus or the likeincludes a general purpose computer once it is programmed to performparticular functions pursuant to instructions from program software.Algorithmic descriptions or symbolic representations are examples oftechniques used by those of ordinary skill in the signal processing orrelated arts to convey the substance of their work to others skilled inthe art. An algorithm is here, and is generally, considered to be aself-consistent sequence of operations or similar signal processingleading to a desired result. In this context, operations or processinginvolve physical manipulation of physical quantities. Typically,although not necessarily, such quantities may take the form ofelectrical or magnetic signals capable of being stored, transferred,combined, compared or otherwise manipulated. It has proven convenient attimes, principally for reasons of common usage, to refer to such signalsas bits, data, values, elements, symbols, characters, terms, numbers,numerals or the like. It should be understood, however, that all ofthese or similar terms are to be associated with appropriate physicalquantities and are merely convenient labels. Unless specifically statedotherwise, as apparent from the following discussion, it is appreciatedthat throughout this specification discussions utilizing terms such as“processing,” “computing,” “calculating,” “determining” or the likerefer to actions or processes of a specific apparatus, such as a specialpurpose computer or a similar special purpose electronic computingdevice. In the context of this specification, therefore, a specialpurpose computer or a similar special purpose electronic computingdevice is capable of manipulating or transforming signals, typicallyrepresented as physical electronic or magnetic quantities withinmemories, registers, or other information storage devices, transmissiondevices, or display devices of the special purpose computer or similarspecial purpose electronic computing device.

Various embodiments of methods and apparatus for the interactiveenhancement of two-dimensional (2D) art with three-dimensional (3D)geometry are described. Embodiments of a surface inflation method, whichmay be implemented as or in a tool, module, plug-in, stand-aloneapplication, etc., may be used to create and modify 3D shapes byinflating the surface that interpolates the input boundaries. Forsimplicity, implementations of embodiments of the surface inflationmethod described herein will be referred to collectively as a surfaceinflation tool. Embodiments may be applied in one or more of, but notlimited to, font design, stroke design, photo enhancement and freeform3D shape design.

Various embodiments may use mean curvature constraints, surface normalconstraints, or a combination of mean curvature constraints and surfacenormal constraints, as boundary conditions to control the inflation. Themean curvature of a surface is an extrinsic measure of curvature thatlocally describes the curvature of the surface. Thus, a mean curvatureconstraint is a specified value for the mean curvature at a particularboundary location, i.e. at a particular point or vertex on an externalor external boundary, or for a particular segment of an external orinternal boundary. A surface normal, or simply normal, to a flat surfaceis a vector perpendicular to that surface. Similarly, a surface normalto a non-flat surface is a vector perpendicular to the tangent plane ata point on the surface. Thus, a surface normal constraint specifiesthat, at this point on the surface (i.e., at a point on an external orinternal boundary of the surface), the surface normal is to point in thespecified direction. As an example, a user may want the surface normalat a point on a boundary to be facing 45 degrees out of plane togenerate a 45 degree bevel, and thus may set the surface normalconstraint to 45 degrees at the point. Surface normal constraint valuesmay be specified at a particular boundary location, i.e. at a particularpoint or vertex on an external or internal boundary, or for a particularsegment of an external or internal boundary.

One embodiment may use mean curvature constraints as boundaryconditions. One embodiment may use surface normal constraints asboundary conditions. One embodiment may use either mean curvatureconstraints or surface normal constraints as boundary conditions. In oneembodiment, both mean curvature constraints and surface normalconstraints may be used as boundary conditions; for example, meancurvature constraints may be used on one portion of a boundary, andsurface normal constraints may be used on another portion of the sameboundary, or on another boundary. The mean curvature constraints and/orsurface normal constraints may be applied to internal or externalboundaries on the surface to be inflated. In embodiments, differentvalues may be specified for the curvature constraints at differentlocations on a boundary. Embodiments may provide one or more userinterface elements via which a user may specify or modify values for theconstraints at locations on boundaries. Embodiments may provide one ormore user interface elements via which a user may add, delete, or modifyexternal or internal boundaries

For examples of external and internal boundaries, see, for example,FIGS. 13 a and 13 b. The outer boundaries of the shaded surface in FIGS.13 a and 13 b are examples of external boundaries. The two white linesover the “eyes” in FIG. 13 b are examples of internal boundaries. Notethat an internal boundary may be a single point, an open line or curve,or a closed boundary. Both external and internal boundaries may beconsidered a type of “constraint” on the surface, as the boundaries arefixed in position during inflation. Thus, internal boundaries andexternal boundaries may be referred to collectively herein as positionconstraints.

In addition to position constraints (external and internal boundaries),mean curvature constraints and surface normal constraints, someembodiments may allow other types of constraints to be specified. Forexample, one embodiment may allow pixel-position constraints to bespecified at points or regions on a surface; the pixel-positionconstraints may be used to limit inflation along a vector. For example,a pixel-position constraint may be used to limit inflation to the zaxis, and thus prevent undesirable shifting of the surface along the xand y axes. As another example, one embodiment may allow orientedposition constraints to be added for surface normal constraints. Someembodiments may also allow arbitrary flow directions to be specified forregions or portions of the surface, or for the entire surface, to bespecified. An example is a gravity option that causes the surface toflow “down”.

Embodiments may leverage characteristics of linear variational surfaceediting techniques to perform the actual inflation, whether meancurvature constraints, surface normal constraints, or both are used.Linear variational surface editing techniques, using some order of thelaplacian operator to solve for smooth surfaces, may provide simplicityand efficiency, but conventionally have not been explored fully forsketch-based modeling interfaces due to limitations. However, as shownherein, these limitations may be leveraged in embodiments to gainadditional degrees of artistic freedom. For some domains of 3Dmodeling—in particular modeling-for-2D (e.g., 3D font design) andpatch-based modeling—embodiments, by leveraging the characteristics oflinear variational surface editing techniques, may provide efficiency,stability, and additional control.

In contrast to conventional methods that use an implicit surfacerepresentation to model surfaces, embodiments may use a polygon (e.g.,triangle) mesh to inflate the given curves. Embodiments may use theinflation metaphor, but without using the chordal axis. Embodiments mayallow both smooth and sharp internal boundaries drawn directly on theinflated surface to modify the surface. In contrast to conventionalmethods, embodiments implement a linear system and work around itsdeficiencies, instead of using a slower, iterative non-linear solverthat is not guaranteed to converge. In addition, embodiments may providea greater range of modeling operations than conventional methods. Whilethis approach may not allow the solution of the whole mesh as a unifiedsystem, embodiments provide an alternative patch-based approach whichmay be more intuitive to users, as the global solve in conventionalmethods may result in surface edits tending to have frustrating globaleffects. While embodiments are generally described as using a trianglemesh, other polygon meshes may be used.

By using the mean curvature value or surface normal value stored at orspecified for boundary vertices as a degree of freedom, embodiments areable to control the inflation of the surface efficiently using a singlelinear system. Embodiments may handle both smooth and sharp positionconstraints. Position constraint vertices may also have curvatureconstraints for controlling the inflation of the local surface.

Typically, the application of shape sketching interfaces is in 3D shapedesign. Conventionally, the two-dimensional (2D) boundaries drawn by theuser are simply a means to get to the end result: the 3D shape. Insteadof using the 2D boundaries to design 3D shapes, embodiments may use theresulting interpolated 3D shapes to enhance the existing 2D boundaries.That is, embodiments may apply the shape-sketching interface to another,interesting application: 2D art creation and design.

Using the inflation metaphor, embodiments may construct a 3D surfacethat interpolates a closed input boundary. Embodiments allow designersto modify the inflated surface with 3D shape modeling features such assharp creases, smooth interpolation curves and local curvature control.Embodiments demonstrate that sophisticated geometric modeling techniquesotherwise found in 3D shape modeling tools can effectively be used todesign interesting looking images. FIG. 1 illustrates an example ofresults of surface inflation using an embodiment of the surfaceinflation tool as described herein. In this example, the word “Hello!”was input as a font outline (the boundary, a position constraint),boundary conditions (mean curvature and/or surface normal constraints)were added, and the surface of the characters was inflated according tothe constraints using an embodiment of the surface inflation method.

Embodiments of the surface inflation tool construct the 2-manifoldsurface that interpolates the input boundary or boundaries. The surfacemay be computed as a solution to a variational problem. The surfaceinflation method implemented by the surface inflation tool may beformulated to solve for the final surface in a single, sparse linearequation, in one embodiment without requiring an additional strip oftriangles at the boundary. In embodiments that employ mean curvatureconstraints, the mean curvature of the vertices on the boundary is adegree of freedom; the surface may be inflated by increasing these meancurvature values. In embodiments that employ surface normal constraints,additional ghost vertices may be used to control the surface normalinternally and at surface boundaries; in this embodiment, the surfacemay be inflated by adjusting the surface normal constraints and thusrotating the boundary's ghost vertices around the boundary. Due to thevariational setup, the inflated surface is smooth except near positionconstraints. The designer can add, remove, or modify internal boundariesand constraints at any time of the design phase, and these internalboundaries and constraints may be smooth or sharp. In one embodiment,the internal boundaries may also have curvature control or surfacenormal control (similar to the external boundaries), thereby allowingthe designer to locally inflate or deflate the surface near the internalboundaries. In one embodiment, external and/or internal boundaries mayhave a stretching control, allowing the designer to extend the regionthat conforms to the target curvature constraint or surface normalconstraint. In one embodiment, constraints may be specified aspixel-position constraints, meaning that they constrain their verticesto always project to the same pixel from some original camera view.

Embodiments of the surface inflation tool may use the inflation metaphorand allow both smooth and sharp position constraints drawn directly onthe inflated surface to modify the surface. Embodiments may use meancurvature constraints and/or surface normal constraints to control theamount of inflation, which allows the problem to be formulated as asingle linear system, unlike conventional methods.

Applications of the surface inflation tool may include one or more of,but are not limited to, font design, stroke design, enhancingphotographs or other images, and modeling 3D shapes from scratch,examples of which are shown in the various Figures. Some embodiments ofthe surface inflation tool may be implemented, for example, as a plug-infor 2D art design tools such as Adobe® Illustrator® and GNU Gimp or as aplug-in for other types of image processing applications. Otherembodiments may be otherwise implemented, for example as a stand-aloneprogram or utility, or as a library function. Various embodiments of thesurface inflation tool may obtain, manipulate, and output digital imagesin any of various digital image formats.

Work Flow

FIGS. 2 a through 2 c illustrate an example of workflow in a surfaceinflation tool according to one embodiment. External boundary curves areinput, as indicated in FIG. 2 a. The flat domain bounded by the inputboundaries (FIG. 2 a) is tessellated. In one embodiment, thetessellation used is triangulation. Boundary constraints (mean curvatureconstraints and/or surface normal constraints) are added, and theresulting surface (FIG. 2 b) is inflated (FIG. 2 c). In this example,the 2D boundaries may, for example, be authored in an art design toolsuch as Adobe® Illustrator and read in as simple poly-lineapproximations. Other methods of obtaining the input 2D boundaries maybe used in various embodiments.

FIG. 3 is a flowchart of a surface inflation method according to oneembodiment. As indicated at 100, the surface inflation tool takes aclosed 2D boundary as input. As indicated at 102, the surface inflationtool tessellates (e.g., triangulates) the area within the externalboundary to generate the initial surface. As indicated at 104, boundaryconstraints (mean curvature constraints and/or surface normalconstraints) may be added to the input boundaries. Other constraintsand/or options, such as pixel position constraints and an arbitrary flowoption, may also be specified for and applied to the surface to beinflated. As indicated at 106, the surface inflation tool then inflatesthe surface according to the specified constraints and options whilemaintaining a fixed boundary for the object being inflated.

Triangulation

As indicated at 102 of FIG. 3, the surface inflation tool tessellatesthe area within the external boundary to generate the initial surface.In performing the tessellation, some embodiments of the surfaceinflation tool may restrict the surface representation to a trianglemesh that is obtained by triangulating the surface within the externalboundary. Any of various triangulation methods may be used to generate ahigh-quality triangulation. Some embodiments may maintain a maximum areaconstraint for the triangles (which may be provided as a configurableoption in the triangulation method) to prevent rendering artifacts dueto very large triangles.

An advantage of solving for the inflated surface using a triangle mesh(as opposed to a regular grid) is efficiency due to mesh adaptivity: thetriangle mesh may be specified to have high detail only near complicatedconstraints, and to be coarse where there are not many constraints. Inone embodiment, the mesh connectivity is not updated as the surface isinflated. In other embodiments, the mesh may dynamically be made denserin parts of the inflated shape that have high curvature, which may bemore efficient and smoother in terms of rendering.

Surface Inflation

As indicated at 106 of FIG. 3, the surface inflation tool inflates thesurface according to the specified constraints while maintaining a fixedboundary. In one embodiment of the surface inflation method, theunconstrained parts of the surface may be obtained by solving avariational system that maintains surface smoothness. Smoothness may bemaintained because it gives an organic look to the inflated surface andremoves any unnatural and unnecessary bumps and creases from thesurface.

In one embodiment, the variational formulation may be based on theprinciples of partial differential equation (PDE) based boundaryconstraint modeling, where the Euler-Lagrange equation of some aestheticenergy functional is solved to yield a smooth surface. One embodimentmay use a ‘thin-plate spline’ as the desired surface; the correspondingEuler-Lagrange equation is the biharmonic equation. In this embodiment,for all free vertices at position x, the PDE Δ²(x)=0 is solved. Thesolution of this PDE yields a C² continuous surface everywhere except atthe position constraints (where the surface can be either C¹ or C⁰continuous). One embodiment may use cotangent-weight baseddiscretization of the laplacian operator Δ(x).

The fourth-order PDE (Δ²(x)=0) may be too slow to solve interactively.Therefore, one embodiment converts the non-linear problem into a linearproblem by assuming that the parameterization of the surface isunchanged throughout the solution. In practice, this means that thecotangent weights used for the Laplacian formulation are computed onlyonce (using the flat, non-inflated surface) and are subsequentlyunchanged as the surface is inflated. This approximation has been usedextensively for constructing smooth shape deformations, but it maysignificantly differ from the correct solution in certain cases. Insteadof correcting this with a slower, sometimes-divergent, iterativenon-linear solver, embodiments may characterize the linear solution anduse its quirks to provide extra dimensions of artist control.

An advantage to using a linear system in the solver is that the linearsystem has a unique solution. In contrast, non-linear systems maygenerate multiple solutions (for example, a global and local minimum).Thus, using a non-linear system, the solver may get trapped in a localminimum, possibly yielding an undesired solution (i.e., the globaloptimum may not be found). Different non-linear solvers may arrive atdifferent local minima. Thus, using a linear system may provideconsistency and efficiency. A trade-off to using a linear system is thatthe resulting surface may not be quite as smooth as a globally optimalsolution to a non-linear system. For artistic purposes, however, asolution produced by a linear system is sufficient.

Linear Systems Mean Curvature Constraint Embodiments

The following describes the formulation of a variational linear systemaccording to embodiments that use mean curvature constraints. In theseembodiments, a linear system A x= b may be implemented, where the matrixA is a sparse n×n matrix (where n is 3× the number of free vertices)that represents the local geometric relationships between the verticesand their neighbors. The vector x of length n represents the positionsof free vertices and the vector b of length n represents the knownquantities. For all three coordinates of every free vertex, an equationis formulated that is linear in terms of the vertex's neighbors. In oneembodiment, the formulation may be based primarily on methods ofdiscrete geometric modeling. A method has been described in the art toformulate a linear system that can handle smooth or sharp internalconstraints; unfortunately the formulation requires a strip of trianglesto complete the one-ring neighborhood of the boundary vertices.Generating this strip of triangles, especially when the boundary curvehas large concavities, is not trivial. In addition, a surface modelingsystem has been described in the art that takes G¹ boundary constraintsand does not need the special triangle strip on the boundary. However,this surface modeling system requires two linear solutions: one for themean curvature scalar field and another for the positions that satisfythe computed mean curvature field. Embodiments of the surface inflationmethod combine the benefits of these two approaches. In so combining,embodiments of the surface inflation method that use the mean curvatureconstraint do not need a strip of triangles on the boundary to performthe inflation, and solve only one linear system. This is possiblebecause the surface inflation method considers the mean curvature at theboundary vertices as a degree of freedom, one that can be used toinflate the surface.

One embodiment may use a conjugate-gradient implementation to solve thelinear system A x= b. Since the matrix A is sparse, symmetric andpositive-definite, other embodiments may factorize the matrix, which maydecrease iterative update times. For example, in one embodiment, aCholesky decomposition of the matrix A may be performed, and in oneembodiment, a direct solver may be used to solve the linear system.Other solvers may be used to solve the linear system in otherembodiments. For example, a sparse Cholesky solver or aconjugate-gradient solver may be used. Other techniques such asmulti-grid solvers may also be used.

FIG. 4 illustrates a mesh vertex of position x and one-ring neighborsy_(i) according to one embodiment. FIG. 4 illustrates various variablesused in the equations below. Vertex x has neighbors y₀,y₁ on theboundary (constrained mean curvature) and neighbors y₂,y₃,y₄ in theinterior with a full one-ring neighborhood (unconstrained meancurvature). The required C² smooth surface can be obtained by solvingfor a surface with a vanishing bi-Laplacian at all vertices:

Δ²(x)=Δ(Δx)=0   (1)

The Laplacian at a mesh vertex is given by its one-ring neighborhood. Adiscrete laplace operator may be used, for example the Laplace-Beltramioperator defined for meshes may be used:

${\Delta \; x} = {\frac{1}{\left( {2A_{x}} \right)}\left( {x - {\sum\limits_{i}{w_{i}y_{i}}}} \right)}$

where w_(i) are the normalized cotangent weights and

$\frac{1}{\left( {2A_{x}} \right)}$

is a scaling term that includes the weighted area A_(x) around thevertex x that improves the approximation of the Laplacian. Note thatother laplace operators for meshes may be used in various embodiments.Substituting in equation (1):

$\begin{matrix}{{\Delta^{2}x} = {{\frac{1}{\left( {2A_{x}} \right)}{\Delta\left( {x - {\sum\limits_{i}{w_{i}y_{i}}}} \right)}} = 0}} & (2)\end{matrix}$

Since Δ is a linear operator:

$\begin{matrix}{{\Delta^{2}x} = {{{\Delta \; x} - {\sum\limits_{i}{w_{i}\Delta \; y_{i}}}} = 0}} & (3)\end{matrix}$

Consider the situation in FIG. 4, where some one-ring neighbors are inthe mesh interior, and some are on the boundary. It is assumed that themean curvature of the boundary vertices is given as a constraint. Assumey_(j) represents the one-ring vertices whose mean curvatures hy_(j) areknown. For those vertices, the Laplacians may be computed simply byusing the expression Δy_(j)=(h_(y) _(j) n_(y) _(j) )/2. Moving suchknown Laplacians to the right hand side of equation (3), the followingis obtained:

$\begin{matrix}{{\Delta^{2}x} = {\left. 0\Rightarrow{{\Delta \; x} - {\sum\limits_{i}{w_{i}\Delta \; y_{i}}}} \right. = {\sum\limits_{j}\frac{w_{j}h_{y_{j}}n_{y_{j}}}{2}}}} & (4)\end{matrix}$

Note that the term (h_(y) _(j) n_(y) _(j) )/2 essentially represents aforce of magnitude 0.5hy_(j) in the direction ny_(j) applied by theneighboring vertex y_(j) on vertex x. In some embodiments, the force isapplied in the direction of the initial vertex normal (the normal in theflat configuration—the Z axis). One embodiment does not use the vertexnormals from the inflated state, as that may produce non-linear vertexmotion that is path-dependent and unintuitive.

Therefore, by increasing the value of hy_(j), the magnitude of the forceon the vertex x is increased, effectively pushing it up.

Finally, the laplacians of vertices with unknown mean curvatures isexpanded in equation (3) to get the linear equation for the free vertexx:

$\begin{matrix}{{x - {\sum\limits_{i}{w_{i}y_{i}}} - {\sum\limits_{i}{w_{i}\left\lbrack {y_{i} - {\sum\limits_{k}{w_{ik}z_{ik}}}} \right\rbrack}}} = {\sum\limits_{i}\frac{w_{j}h_{y_{j}}n_{y_{j}}}{2}}} & (5)\end{matrix}$

Constructing such equations for every free vertex yields the linearsystem A x= b, the solution of which provides the inflated surface.

Surface Normal Constraint Embodiments

The following describes the formulation of a variational linear systemaccording to embodiments that use surface normal constraints. Inembodiments, a linear system A x= b may be implemented, where the matrixA is a sparse n×n matrix (where n is 3× the number of free vertices)that represents the local geometric relationships between the verticesand their neighbors. The vector x of length n represents the positionsof free vertices and the vector b of length n represents the knownquantities. For all three coordinates of every free vertex, an equationis formulated that is linear in terms of the vertex's neighbors. In thisembodiment, the formulation may be based primarily on a method ofdiscrete geometric modeling, with the addition of constraint types. Amethod has been described in the art to formulate a linear system thatcan handle smooth or sharp internal constraints; unfortunately, theformulation requires a strip of triangles to complete the one-ringneighborhood of the boundary vertices. Generating this strip oftriangles, especially when the boundary curve has large concavities, isnot trivial. Therefore, embodiments of the surface inflation tool thatuse a surface normal constraint provide a method to “fake” thesetriangles with locally-correct “ghost” vertices.

FIG. 5 illustrates a mesh vertex of position x and one-ring neighborsy_(i) according to one embodiment. FIG. 5 shows various variables usedin the equations below. Vertex x has neighbors y₀,y₁ on the boundary,and neighbors y₂,y₃,y₄ in the interior with a full one-ringneighborhood. Each boundary vertex has its own ghost vertex and twocorresponding ghost triangles—for example, y₁ has ghost vertex g—toartificially provide the required one-ring neighborhood.

Consider a mesh vertex of position x and one-ring neighbors y_(i) asshown in FIG. 5. A C² smooth surface can be obtained by solving for asurface with a vanishing bi-Laplacian at all vertices:

Δ²(x)=Δ(Δx)=0   (6)

The Laplacian at a mesh vertex is given by its one-ring neighborhood:

${\Delta \; x} = {\sum\limits_{i}{w_{i}\left( {x - y_{i}} \right)}}$

where w_(i) are the unnormalized cotangent weights scaled by inversevertex area. Substituting in equation (6):

$\begin{matrix}{{\Delta^{2}x} = {{\Delta\left( {\sum\limits_{i}{w_{i}\left( {x - y_{i}} \right)}} \right)} = 0}} & (7)\end{matrix}$

Since Δ is a linear operator:

$\begin{matrix}{{\Delta^{2}x} = {{{\sum\limits_{i}{w_{i}\Delta \; x}} - {\sum\limits_{i}{w_{i}\Delta \; y_{i}}}} = 0}} & (8)\end{matrix}$

This expands finally to:

$\begin{matrix}{\begin{matrix}{{\left( {\sum\limits_{i}w_{i}} \right)^{2}x} - {\left( {\sum\limits_{i}w_{i}} \right)\left( {\sum\limits_{i}{w_{i}y_{i}}} \right)} -} \\{\sum\limits_{i}{w_{i}\left\lbrack {{\left( {\sum\limits_{k}w_{ik}} \right)y_{i}} - {\sum\limits_{k}{w_{ik}z_{ik}}}} \right\rbrack}}\end{matrix} = 0} & (9)\end{matrix}$

where z_(ik) refers to ghost vertices where necessary to complete aone-ring. In one embodiment, constrained vertices may be treated asabsolute constraints, so their positions are moved to the right handside of the system. Because it may be convenient to over-constrain thesystem, and satisfy other types of constraints in a least squares sense,in one embodiment the whole equation may be scaled by the inverse of:

$\left( {\sum\limits_{i}w_{i}} \right)^{2}$

so that the magnitude of errors will be proportional to a difference inpositions, and not scaled by any area or mean curvature values.Constructing such equations for every free vertex gives the linearsystem A x= b, whose solution provides the inflated surface. Since theconstruction is not symmetric and may be over-constrained, it may besolved using the normal equations.

Placement of Ghost Vertices

In one embodiment, for each patch of a mesh surface, a canonical viewdirection may be assumed to be known; this may be the direction fromwhich, for example, an original photograph was taken, or from which theoriginal boundary constraints were drawn. An ‘up’ vector which pointstowards the camera of this canonical view may be derived. Ghost verticesmay then be placed in a plane perpendicular to the ‘up’ vector, andnormal to the constraint curve of their parent vertex. In oneembodiment, each ghost vertex may be placed the same fixed distance dfrom the curve. For example, in FIG. 5, assuming a vector out of thepage u, the ghost vertex g is positioned at:

y ₁ +d(u×(z ₂₁ −y ₀))/∥u×(z ₂₁ −y ₀)∥

The ghost vertices may then be rotated about the tangent of theconstraint curve (the boundary) to change the normal direction.

Note that ghost vertices may be added for both external and internalboundaries.

Internal Constraints

In embodiments, the user may draw internal boundaries as internalposition constraints anywhere on an inflated surface to automaticallyobtain the new inflated surface with the new internal boundaries inplace. The user may also specify boundary constraints (mean curvatureand/or surface normal constraints) for the internal boundaries. Uponadding a new internal boundary, the linear system A x= b needs to bere-initialized and solved. FIG. 6 is a general flowchart of a method forre-initializing and solving the linear system, according to embodiments.The re-solving may be performed differently for internal boundaries withmean curvature constraints than for internal boundaries with surfacenormal constraints.

The existing surface (without the new internal boundary) is flattened bychanging the boundary vertex mean curvatures to zero and moving allinternal position constraints to their 2D positions, as indicated at200. The 2D positions of the new internal boundary vertices are computedby using the barycentric coordinates within the flattened triangle mesh,as indicated at 202. The area bounded by the external boundary istessellated (e.g., triangulated) subject to the 2D positions of theinternal boundaries, as indicated at 204. The resulting surface isre-inflated, as indicated at 206. In one embodiment, for mean curvatureconstraints, this may be performed by setting the boundary vertex meancurvatures to the original values and moving the position constraintcurves to their old positions. In one embodiment, for surface normalconstraints, the resulting surface may be re-inflated by moving theposition constraint curves to their old positions and re-solving thesystem for the positions of the free vertices.

FIGS. 7 a through 7 d graphically illustrate an example that shows theabove methods for adding internal boundaries according to embodiments.Suppose the user wants a sharp internal boundary. The user may draw aninternal boundary on the inflated surface, as illustrated in FIG. 7 a.The user may add mean curvature constraints and/or surface normalconstraints to the new internal boundary. The user may also add, remove,or modify mean curvature constraints and/or surface normal constraintsto the external boundary if desired. Other constraints, such as pixelposition constraints, may also be added to, removed from, or modifiedfor the surface. The surface inflation tool flattens the inflatedsurface and computes the 2D position of the internal boundary, asillustrated in FIG. 7 b. The surface inflation tool then re-tessellatesthe domain subject to the new internal boundary, as illustrated in FIG.7 c, re-inflates the surface, and in one embodiment moves the internalboundary to its original location to generate the new inflated surfaceas illustrated in FIG. 7 d. In one embodiment, the operations shown inFIGS. 7 b and 7 c may be performed transparently to the user.

Smoothness of Position Constraints

In one embodiment, either smooth (C¹) or sharp (C⁰) position constraintsmay be specified. The smoothness value may be varied by assigning aweight to the constrained vertex in equation (5) or in equation (9). Inone embodiment, the weight that controls the smoothness of the positionconstraints may take any floating point value between 0 (C⁰ continuous)and 1 (C¹ continuous). However, in one embodiment, it may be useful tohave only two options (smooth/sharp), and to draw position constraintswith a fixed smoothness for all vertices. Other embodiments may allowthe use of varying smoothness across individual position constraints.FIGS. 14 a through 17 c shows some examples of smooth/sharp positionconstraints.

Curvature Constraints

In one embodiment, curvature constraints may be specified along withposition constraints. When the value of a curvature constraint ismodified, the surface is modified so that the approximation of the meancurvature at the constraint point matches the value of the curvatureconstraint. The curvature constraint may be used to locally inflate ordeflate the surface around the position-constrained vertices. As such,embodiments may provide a sketch-based modeling gesture. In oneembodiment, the initial value for the curvature constraint is set tozero, but in other embodiments the initial value may be set to anyarbitrary value.

Options for Constraints

Assigning a mean curvature constraint or a surface normal constraint toa vertex is an indirect method of applying a force to their one-ringneighbors along the direction perpendicular to the initial, flatsurface. However, in some embodiments, the default behavior may bemodified, and additional forces may be applied in arbitrary directions.As an example, in one embodiment, a ‘gravity’ option may be added to thecurvature constraints where another force is applied in a slightlydownward direction (to the right hand side of equation (5)), causing theentire surface to bend downwards. This may be used, for example, tocreate the illusion of a viscous material on a vertical plane. See, forexample, FIG. 11 b and FIG. 13 h. In some embodiments, other directionsthan “down” may be specified to cause the surface to flow in a specifieddirection.

Oriented Position Constraints

In one embodiment, the ghost vertex concept described for surface normalconstraints may be extended to internal boundaries that may be used tocontrol the orientation of the surface along the internal boundaries. Inone embodiment, to do so, the way the laplacian at the constrainedvertex is calculated may be modified, as shown in FIGS. 8 a through 8 c.As seen in FIGS. 8 a through 8 c, the ghost laplacians extend naturallyto internal boundaries except at the endpoints of open internalboundaries; here there are no longer well defined sides of the curve(especially for internal boundaries which are just a single point) andtherefore a different method should be used for computing the laplacian.In one embodiment, the laplacian in this degenerate case may be definedusing just the originating vertex x for which the bilaplacian is beingcomputed and the opposing ghost vertex g as the “‘one ring”’ of theconstrained vertex. Since the measures of vertex area and cotangentweights do not extend to this case, the sum of area-normalized cotangentweights from the originating vertex may be used. The method then letsthe two vertices share that weight sum equally. Therefore, the laplacianmay be defined as:

${\left( {w_{i}/{\sum\limits_{i}w_{i}}} \right)y_{i}} - {\left( {g + x} \right){w_{i}/\left( {2{\sum\limits_{i}w_{i}}} \right)}}$

FIGS. 8 a through 8 c illustrate oriented position constraints accordingto one embodiment. An internal oriented position constraint line isshown in FIG. 8 a. When calculating the bilaplacian of vertex x, oneembodiment may calculate the laplacian at y by creating ghost g, so theone ring of vertex y is (x, n1, g, n2) as shown in FIG. 8 b. Note thatn₁ is on the end of the constraint line, so to compute its laplacian gis instead placed along the vector from x to n₁, and the laplacian at n₁is computed using only vertices x, n₁, and g.

Pixel-Position Constraints

FIG. 9 illustrates pixel-position constraints according to oneembodiment. Referring to FIG. 9, a pixel-position constraint allows avertex p to move freely along vector d. A user may wish to constrain avertex (or a region) on the surface to always project to the sameposition in screen space from a given view, without fully constrainingthe position of that vertex or region. In one embodiment, this may beallowed by over-constraining the linear system with additionalconstraints referred to as pixel-position constraints. These constraintsmay be written into the matrix as two linear constraints for twoarbitrary unit vectors orthogonal to the camera ray d—o₁ and o₂. For apoint p with initial position p′, the constraint equations areo₁·(p−p′)=0, and the equivalent for o₂.

Note that without pixel-position constraints, the linear system may bewritten separately for x, y and z, but for arbitrary pixel positionconstraints, x, y and z may be arbitrarily coupled. This may have aperformance cost, as the matrix would be nine times larger. For lessfree-form applications, it may therefore be useful to keep the systemdecoupled by implementing pixel-position constraints only foraxis-aligned orthogonal views. In these cases the constraint is simplyimplemented by fixing the vertex coordinates in two dimensions andleaving it free in the third.

Pixel position constraints may be used with mean curvature constraints,with surface normal constraints, or with a combination of mean curvatureconstraints and surface normal constraints.

Mixing Pixel-Position and Orientation Constraints

In many cases, orientation and pixel-position are known, but it may notbe desired by the artist to fix the position fully—for example, whenmodeling a face, there may be a certain angle at the nose, but theartist may still want to allow the nose to move smoothly out whenpuffing out the cheeks of the character. To allow this, one embodimentmay mix pixel-position and orientation constraints. The vertex loses itsbilaplacian smoothness constraints, and gains ghost vertices andpixel-position constraints. Ghost vertices are specified relative to thefree vertices of the pixel-position constraint, instead of absolutely.However, this removes three bilaplacian constraint rows in the matrixfor every two pixel-position rows it adds (assuming a coupled system)making the system underconstrained. Therefore, additional constraintsmay be needed. In one embodiment, for a first additional constraint, itmay be observed that when a user draws a line of pixel-positionconstraints, they likely want the line to retain some smoothness ororiginal shape. For adjacent vertices p₁, p₂ on the constraint line,which are permitted to move along vectors d₁ and d₂ respectively, oneembodiment may therefore constrain the vertices to satisfy:

(p ₁ −p ₂)·(d ₁ +d ₂)/2=0

Since the system is still one constraint short, one embodiment may addan additional constraint specifying that the laplacian at the endpointsof the constraint line (computed without any ghost vertices) shouldmatch the expected value (which is known by the location of the ghostvertices relative to the constraint curve). Scaling these laplacianconstraints adjusts the extent to which the constrained vertices move tosatisfy the normal constraints.

Exploiting the Linearization

The system described herein is a linear system because the non-lineararea and cotangent terms have been made constant, as calculated in someoriginal configuration. The linearization may be thought of as allowingthe ‘stretch’ of triangles to be considered as curvature, in addition toactual curvature; therefore variation is minimized in trianglestretch+curvature, instead of just curvature. In some embodiments, thiscan be exploited by intentionally stretching triangles: for example, byintentionally moving ghost vertices, their area of effect may beincreased. This is similar to moving a bezier control point along thetangent of a curve.

The linearization may also cause the solution to collapse to a plane ifall of the control vertices are coplanar. This may be visible in thesystem as the ghost vertices at the boundaries are rotated to becoplanar and inside the shape, resulting in a flat, folded shape.However, in one embodiment, the need for modeling this shape with asingle linear system may be avoided by allowing the system to be used asa patch-based modeling system, with ghost vertices enforcing C¹continuity across patch boundaries.

Applications of Embodiments

The following describes some examples of applications of embodiments ofthe surface inflation tool, and is not intended to be limiting.

3D Font Design

An application of embodiments of the surface inflation tool may be infont design. The outline of a font character may be inflated to providedepth to the font. Moreover, the shape of the inflated character can becontrolled and enhanced by adding smooth or sharp position constraintsand boundary constraints (either mean curvature constraints, surfacenormal constraints, or both).

FIGS. 10 a through 10 e illustrate an example of 3D font designaccording to one embodiment. The letter ‘A’ is read in as a pair ofclosed boundary curves (the external boundary), as illustrated in FIG.10 a. By increasing the mean curvature at the constrained boundaryvertices, the surface bounded by the input curves is inflated, asillustrated in FIG. 10 b. Next, internal offset curves are read in ascurvature and sharp position constraints, as illustrated in FIG. 10 c.The internal curves are raised to give a beveled effect, as illustratedin FIG. 10 d, and then made sharper by increasing the curvature at thecurve vertices, as illustrated in FIG. 10 e.

Stroke Design

In addition to inflating already complete input 2D curves (such as afont outline), embodiments of the surface inflation tool may be used asa tool for inflating 2D elements as they are generated. One example isthat of strokes. Currently, art tools such as Adobe® Illustrator®support strokes with a variety of brush shapes, thickness, and incidentangles. Embodiments may allow the addition of another option to astroke: depth. Varying stroke depth may be implemented in embodiments bychanging the mean curvature or the surface normal at the strokeboundary. Moreover, in one embodiment, the medial axis of the stroke maybe provided as a constraint curve, further increasing the range ofstroke shapes possible.

FIGS. 11 a through 11 d show an example of stroke design according toone embodiment. The surface inflation tool is used to inflate theoutline of a paintbrush stroke, as illustrated in FIG. 11 a. The gravityoption may be selected to produce an effect of paint drip, asillustrated in FIG. 11 b. The stroke axis is added as a sharp positionconstraint, as illustrated in FIG. 11 c. A grooved stroke is produced byinflating the rest of the surface, as illustrated in FIG. 11 d.

Photograph Inflation

Another application of embodiments of the surface inflation tool may bein photograph inflation, or more generally digital image inflation. Forexample, using the surface inflation tool, a user can interactively adddepth to an input photograph or other digital image by drawing andmanipulating position constraints on the image. As another example, atool such as a boundary tracking or object location tool may be used toautomatically locate boundaries on an image, e.g. a digital photograph,and the boundaries may be input to the surface inflation tool asposition constraints. The position constraints can be smooth (for imagescontaining rounded edges), sharp (images of rectilinear shapes), or acombination thereof. Mean curvature constraints, surface normalconstraints, or both types of boundary constraints may be specified forthe boundaries on the image.

FIGS. 12 a and 12 b show an example of photograph inflation according toone embodiment. In the example, the surface inflation tool is used toinflate a digital photograph of the Taj Mahal. Position constraintcurves of varying types (e.g., smooth, sharp, and curvature constrained)are specified, as illustrated by the white lines in FIG. 12 a, to definethe inflated shape. In FIG. 12 a, the solid white lines represent sharpposition constraints, while the black dotted lines represent smoothposition constraints. A sharp position constraint is where the surfacepasses through the position constraint curve, but does not maintaingeometric smoothness across the position constraint curve. A smoothposition constraint is where the surface passes through the positionconstraint curve while maintaining smoothness across the positionconstraint curve. The squares associated with the dotted lines representlocations that have been selected to specify the position constraints.For example, two squares at the end of a line represent locations thatwere selected to specify the line between the squares. In oneembodiment, the locations represented by the squares may be selected bymouse-clicking at the locations. As an example, in the top-most dome ofthe Taj Mahal, four different points of a smooth curve (dotted lines)with different depth levels produce the inflated dome shape. The sharpposition constraint curves surrounding the dome (solid white lines)prevent the inflation from propagating beyond the dome. After thedesired constraints are specified, the image may be inflated using thesurface inflation method as described herein to generate an inflatedimage, as illustrated in FIG. 12 b.

3D Shape Modeling

Embodiments of the surface inflation tool may be used in the freeformdesign of arbitrary 3D shapes. By adding smooth and/or sharp positionconstraints and boundary constraints, an inflated shape may be modified.Mean curvature constraints, surface normal constraints, or bothconstraints may be used.

FIGS. 13 a through 13 h graphically illustrate freeform 3D shape designaccording to one embodiment. Given the outline of a cartoon face (FIG.13 a), the interior is inflated, and two smooth position constraints areadded near the eyebrows (FIG. 13 b). One of the eyebrows is pulled upand the other is pulled down (FIG. 13 c). A sharp position constraint isadded near the mouth (FIG. 13 d) and the nearby surface is inflated bymodifying the mean curvature constraints or the surface normalconstraints (FIG. 13 e). A smooth position constraint is added near thebridge of the nose (FIG. 13 f) to get the final surface (FIG. 13 g). Agravity or other directional flow may optionally be added (see abovediscussion) for this 3D shape to create a directional flow effect (FIG.13 h).

Surface Normal Constraint Examples

FIGS. 14 a through 16 b illustrate examples of the application ofsurface normal constraints, and also show the effect of smooth positionconstraints, according to some embodiments. These examples show thatsurface normal constraints may be used (rotated) to generate concave,flat, or convex angles at boundaries.

FIGS. 14 a and 14 b show an example surface generated with a smoothposition constraint at the position indicated by the “+”, and with aconcave angle at the boundary as specified using surface normalconstraints, according to one embodiment. The arrows in FIG. 14 bindicate the direction of the surface normal at the external boundary ofthe surface.

FIGS. 15 a and 15 b show an example surface generated with a smoothposition constraint at the position indicated by the “+”, and with aflat angle at the boundary as specified using surface normalconstraints, according to one embodiment. The arrows in FIG. 15 bindicate the direction of the surface normal at the external boundary ofthe surface.

FIGS. 16 a and 16 b show an example surface generated with a smoothposition constraint at the position indicated by the “+”, and with aconvex angle at the boundary as specified using surface normalconstraints, according to one embodiment. The arrows in FIG. 16 bindicate the direction of the surface normal at the external boundary ofthe surface.

FIGS. 17 a through 17 c illustrate modifying the angle(s) at an internalboundary of an example surface using surface normal constraints,according to some embodiments. The internal boundary in the images isrepresented by the black-and-white dashed line. FIG. 17 a shows flatangles across the internal boundary. FIG. 17 b shows sharp angles acrossthe internal boundary. FIG. 17 c shows that the surface is locallyinflated with varying angles at the internal boundary.

Implementations

FIG. 18 illustrates an example embodiment of a surface inflation toolthat implements a surface inflation method as described herein. As notedabove, applications of the surface inflation tool 300 may include one ormore of, but are not limited to, font design, stroke design, enhancingphotographs, and modeling 3D shapes from scratch. Some embodiments ofthe surface inflation tool 300 may be implemented, for example, as aplug-in for 2D art design tools such as Adobe® Illustrator® and GNUGimp. Other embodiments may be otherwise implemented, for example as astand-alone program or utility.

Surface inflation tool 300 may provide a user interface 302 thatprovides one or more textual and/or graphical user interface elements,modes or techniques via which a user may enter, modify, indicate orselect images, or regions of images, to be inflated (represented byinput image 310), enter, modify, or select position constraints, selecta gravity option or similar arbitrary directional flow option, input ordraw strokes into shapes, images, or regions of digital images, specifyor modify smooth or sharp position and boundary constraints includingmean curvature constraints and surface normal constraints, specifypixel-position constraints, and in general provide input to and/orcontrol various aspects of surface inflation using embodiments of asurface inflation tool 300 as described herein. In one embodiment,surface inflation tool 300 may provide real-time or near-real-timefeedback to the user via dynamic display on a display device(s) ofmodifications to the target image 310 made according to the user inputvia user interface 302. Thus, the user may make additional input ormanipulation of the image or shape using the surface inflation tool 300,as illustrated by intermediate image(s) 320. Results are output as thefinal image 322. Final image 322 (as well as intermediate image(s) 320)may be displayed on a display device, printed, and/or written to orstored on any of various types of memory media, such as storage media orstorage devices.

Example System

Various components of embodiments of a surface inflation tool asdescribed herein may be executed on one or more computer systems, whichmay interact with various other devices. One such computer system isillustrated by FIG. 19. In the illustrated embodiment, computer system700 includes one or more processors 710 coupled to a system memory 720via an input/output (I/O) interface 730. Computer system 700 furtherincludes a network interface 740 coupled to I/O interface 730, and oneor more input/output devices 750, such as cursor control device 760,keyboard 770, and display(s) 780. In some embodiments, it iscontemplated that embodiments may be implemented using a single instanceof computer system 700, while in other embodiments multiple suchsystems, or multiple nodes making up computer system 700, may beconfigured to host different portions or instances of embodiments. Forexample, in one embodiment some elements may be implemented via one ormore nodes of computer system 700 that are distinct from those nodesimplementing other elements.

In various embodiments, computer system 700 may be a uniprocessor systemincluding one processor 710, or a multiprocessor system includingseveral processors 710 (e.g., two, four, eight, or another suitablenumber). Processors 710 may be any suitable processor capable ofexecuting instructions. For example, in various embodiments, processors710 may be general-purpose or embedded processors implementing any of avariety of instruction set architectures (ISAs), such as the x86,PowerPC, SPARC, or MIPS ISAs, or any other suitable ISA. Inmultiprocessor systems, each of processors 710 may commonly, but notnecessarily, implement the same ISA.

In some embodiments, at least one processor 710 may be a graphicsprocessing unit. A graphics processing unit or GPU may be considered adedicated graphics-rendering device for a personal computer,workstation, game console or other computer system. Modern GPUs may bevery efficient at manipulating and displaying computer graphics, andtheir highly parallel structure may make them more effective thantypical CPUs for a range of complex graphical algorithms. For example, agraphics processor may implement a number of graphics primitiveoperations in a way that makes executing them much faster than drawingdirectly to the screen with a host central processing unit (CPU). Invarious embodiments, the methods disclosed herein for surface inflationmay be implemented by program instructions configured for execution onone of, or parallel execution on two or more of, such GPUs. The GPU(s)may implement one or more application programmer interfaces (APIs) thatpermit programmers to invoke the functionality of the GPU(s). SuitableGPUs may be commercially available from vendors such as NVIDIACorporation, ATI Technologies, and others.

System memory 720 may be configured to store program instructions and/ordata accessible by processor 710. In various embodiments, system memory720 may be implemented using any suitable memory technology, such asstatic random access memory (SRAM), synchronous dynamic RAM (SDRAM),nonvolatile/Flash-type memory, or any other type of memory. In theillustrated embodiment, program instructions and data implementingdesired functions, such as those described above for a surface inflationtool, are shown stored within system memory 720 as program instructions725 and data storage 735, respectively. In other embodiments, programinstructions and/or data may be received, sent or stored upon differenttypes of computer-accessible media or on similar media separate fromsystem memory 720 or computer system 700. Generally speaking, acomputer-accessible medium may include storage media or memory mediasuch as magnetic or optical media, e.g., disk or CD/DVD-ROM coupled tocomputer system 700 via I/O interface 730. Program instructions and datastored via a computer-accessible medium may be transmitted bytransmission media or signals such as electrical, electromagnetic, ordigital signals, which may be conveyed via a communication medium suchas a network and/or a wireless link, such as may be implemented vianetwork interface 740.

In one embodiment, I/O interface 730 may be configured to coordinate I/Otraffic between processor 710, system memory 720, and any peripheraldevices in the device, including network interface 740 or otherperipheral interfaces, such as input/output devices 750. In someembodiments, I/O interface 730 may perform any necessary protocol,timing or other data transformations to convert data signals from onecomponent (e.g., system memory 720) into a format suitable for use byanother component (e.g., processor 710). In some embodiments, I/Ointerface 730 may include support for devices attached through varioustypes of peripheral buses, such as a variant of the Peripheral ComponentInterconnect (PCI) bus standard or the Universal Serial Bus (USB)standard, for example. In some embodiments, the function of I/Ointerface 730 may be split into two or more separate components, such asa north bridge and a south bridge, for example. In addition, in someembodiments some or all of the functionality of I/O interface 730, suchas an interface to system memory 720, may be incorporated directly intoprocessor 710.

Network interface 740 may be configured to allow data to be exchangedbetween computer system 700 and other devices attached to a network,such as other computer systems, or between nodes of computer system 700.In various embodiments, network interface 740 may support communicationvia wired or wireless general data networks, such as any suitable typeof Ethernet network, for example; via telecommunications/telephonynetworks such as analog voice networks or digital fiber communicationsnetworks; via storage area networks such as Fibre Channel SANs, or viaany other suitable type of network and/or protocol.

Input/output devices 750 may, in some embodiments, include one or moredisplay terminals, keyboards, keypads, touchpads, scanning devices,voice or optical recognition devices, or any other devices suitable forentering or retrieving data by one or more computer system 700. Multipleinput/output devices 750 may be present in computer system 700 or may bedistributed on various nodes of computer system 700. In someembodiments, similar input/output devices may be separate from computersystem 700 and may interact with one or more nodes of computer system700 through a wired or wireless connection, such as over networkinterface 740.

As shown in FIG. 19, memory 720 may include program instructions 725,configured to implement embodiments of a surface inflation tool asdescribed herein, and data storage 735, comprising various dataaccessible by program instructions 725. In one embodiment, programinstructions 725 may include software elements of a surface inflationtool as illustrated in the above Figures. Data storage 735 may includedata that may be used in embodiments. In other embodiments, other ordifferent software elements and data may be included.

Those skilled in the art will appreciate that computer system 700 ismerely illustrative and is not intended to limit the scope of a surfaceinflation tool as described herein. In particular, the computer systemand devices may include any combination of hardware or software that canperform the indicated functions, including computers, network devices,internet appliances, PDAs, wireless phones, pagers, etc. Computer system700 may also be connected to other devices that are not illustrated, orinstead may operate as a stand-alone system. In addition, thefunctionality provided by the illustrated components may in someembodiments be combined in fewer components or distributed in additionalcomponents. Similarly, in some embodiments, the functionality of some ofthe illustrated components may not be provided and/or other additionalfunctionality may be available.

Those skilled in the art will also appreciate that, while various itemsare illustrated as being stored in memory or on storage while beingused, these items or portions of them may be transferred between memoryand other storage devices for purposes of memory management and dataintegrity. Alternatively, in other embodiments some or all of thesoftware components may execute in memory on another device andcommunicate with the illustrated computer system via inter-computercommunication. Some or all of the system components or data structuresmay also be stored (e.g., as instructions or structured data) on acomputer-accessible medium or a portable article to be read by anappropriate drive, various examples of which are described above. Insome embodiments, instructions stored on a computer-accessible mediumseparate from computer system 700 may be transmitted to computer system700 via transmission media or signals such as electrical,electromagnetic, or digital signals, conveyed via a communication mediumsuch as a network and/or a wireless link. Various embodiments mayfurther include receiving, sending or storing instructions and/or dataimplemented in accordance with the foregoing description upon acomputer-accessible medium. Accordingly, the present invention may bepracticed with other computer system configurations.

CONCLUSION

Various embodiments may further include receiving, sending or storinginstructions and/or data implemented in accordance with the foregoingdescription upon a computer-accessible medium. Generally speaking, acomputer-accessible medium may include storage media or memory mediasuch as magnetic or optical media, e.g., disk or DVD/CD-ROM, volatile ornon-volatile media such as RAM (e.g. SDRAM, DDR, RDRAM, SRAM, etc.),ROM, etc., as well as transmission media or signals such as electrical,electromagnetic, or digital signals, conveyed via a communication mediumsuch as network and/or a wireless link.

The various methods as illustrated in the Figures and described hereinrepresent examples of embodiments of methods. The methods may beimplemented in software, hardware, or a combination thereof. The orderof method may be changed, and various elements may be added, reordered,combined, omitted, modified, etc.

Various modifications and changes may be made as would be obvious to aperson skilled in the art having the benefit of this disclosure. It isintended that the invention embrace all such modifications and changesand, accordingly, the above description to be regarded in anillustrative rather than a restrictive sense.

1. A computer-implemented method, comprising: tessellating a surfacebounded by a closed two-dimensional boundary to generate an initialtessellated surface, wherein the initial tessellated surface is boundedby the boundary; specifying a mean curvature constraint value at one ormore boundary vertices of the initial tessellated surface; inflating theinitial tessellated surface according to the specified mean curvatureconstraint values at the one or more boundary vertices while maintainingthe boundary to generate an inflated surface that provides athree-dimensional geometric effect; and displaying the boundary and thethree-dimensional geometric effect as an inflated three-dimensionalobject.
 2. The computer-implemented method as recited in claim 1,wherein said tessellating comprises triangulating the surface bounded bythe closed two-dimensional boundary, and wherein the initial tessellatedsurface is a triangle mesh.
 3. The computer-implemented method asrecited in claim 1, further comprising: specifying an internal boundaryon the inflated surface; and generating a new inflated surface thatprovides a different three-dimensional geometric effect according to theinternal boundary and the specified mean curvature constraint values atthe one or more boundary vertices.
 4. The computer-implemented method asrecited in claim 3, wherein said generating a new inflated surfacecomprises: flattening the initial inflated surface; computing the 2Dposition of the internal boundary; tessellating the surface subject tothe internal boundary to generate a new tessellated surface; specifyinga mean curvature constraint value at one or more vertices of the newtessellated surface at the internal boundary; and inflating the newtessellated surface according to the specified mean curvature constraintvalues at the one or more boundary vertices and the mean curvatureconstraint values at the one or more vertices of the new tessellatedsurface at the internal boundary.
 5. The computer-implemented method asrecited in claim 4, further comprising moving the internal boundary toits original 2D position.
 6. The computer-implemented method as recitedin claim 3, wherein said specifying an internal boundary on the inflatedsurface comprises receiving user input via a user interface, wherein theuser input indicates the internal boundary on the displayed inflatedthree-dimensional object.
 7. The computer-implemented method as recitedin claim 1, wherein said specifying a mean curvature constraint value atone or more boundary vertices of the initial tessellated surface furthercomprises receiving user input indicating the mean curvature constraintvalues at the one or more boundary vertices.
 8. The computer-implementedmethod as recited in claim 1, further comprising: specifying a flowdirection for the surface bounded by the closed two-dimensionalboundary; and inflating the initial tessellated surface according to thespecified mean curvature constraint values at the one or more boundaryvertices and the flow direction for the surface while maintaining theboundary to generate an inflated surface that provides athree-dimensional geometric effect, wherein the three-dimensionalgeometric effect flows in the specified flow direction.
 9. Thecomputer-implemented method as recited in claim 1, wherein saidinflating the initial tessellated surface is performed according to alinear system.
 10. The computer-implemented method as recited in claim9, wherein said linear system does not require the tessellation toextend beyond the boundary.
 11. The computer-implemented method asrecited in claim 1, wherein said inflating the initial tessellatedsurface is performed by solving a single linear system.
 12. Thecomputer-implemented method as recited in claim 1, further comprising:specifying a position constraint on the inflated surface; specifying asmoothness value for the position constraint, wherein the smoothnessvalue indicates a sharp position constraint or a smooth positionconstraint; and generating a new inflated surface that provides adifferent three-dimensional geometric effect according to the positionconstraint and the specified smoothness value, wherein the new inflatedsurface passes through the position constraint but does not maintaingeometric smoothness across the position constraint if the smoothnessvalue indicates a sharp position constraint, and wherein the newinflated surface passes through the position constraint whilemaintaining smoothness across the position constraint curve if thesmoothness value indicates a smooth position constraint.
 13. A system,comprising: at least one processor; a display device; and a memorycomprising program instructions, wherein the program instructions areexecutable by the at least one processor to implement a surfaceinflation tool configured to: tessellate a surface bounded by a closedtwo-dimensional boundary to generate an initial tessellated surface,wherein the initial tessellated surface is bounded by the boundary;specify a mean curvature constraint value at one or more boundaryvertices of the initial tessellated surface; inflate the initialtessellated surface according to the specified mean curvature constraintvalues at the one or more boundary vertices while maintaining theboundary to generate an inflated surface that provides athree-dimensional geometric effect; and display the boundary and thethree-dimensional geometric effect as an inflated three-dimensionalobject on the display device.
 14. The system as recited in claim 13,wherein, to tessellate, the surface inflation tool is configured totriangulate the surface bounded by the closed two-dimensional boundary,and wherein the initial tessellated surface is a triangle mesh.
 15. Thesystem as recited in claim 13, wherein the surface inflation tool isconfigured to: specify an internal boundary on the inflated surface; andgenerate a new inflated surface that provides a differentthree-dimensional geometric effect according to the internal boundaryand the specified mean curvature constraint values at the one or moreboundary vertices.
 16. The system as recited in claim 15, wherein, togenerate a new inflated surface, the surface inflation tool isconfigured to: flatten the initial inflated surface; compute the 2Dposition of the internal boundary; tessellate the surface subject to theinternal boundary to generate a new tessellated surface; specify a meancurvature constraint value at one or more vertices of the newtessellated surface at the internal boundary; and inflate the newtessellated surface according to the specified mean curvature constraintvalues at the one or more boundary vertices and the mean curvatureconstraint values at the one or more vertices of the new tessellatedsurface at the internal boundary.
 17. The system as recited in claim 16,wherein the surface inflation tool is further configured to move theinternal boundary to its original 2D position.
 18. The system as recitedin claim 15, wherein, to specify an internal boundary on the inflatedsurface, the surface inflation tool is configured to receive user inputvia a user interface, wherein the user input indicates the internalboundary on the displayed inflated three-dimensional object.
 19. Thesystem as recited in claim 13, wherein, to specify a mean curvatureconstraint value at one or more boundary vertices of the initialtessellated surface, the surface inflation tool is configured to receiveuser input indicating the mean curvature constraint values at the one ormore boundary vertices.
 20. The system as recited in claim 13, whereinthe surface inflation tool is configured to: specify a flow directionfor the surface bounded by the closed two-dimensional boundary; andinflate the initial tessellated surface according to the specified meancurvature constraint values at the one or more boundary vertices and theflow direction for the surface while maintaining the boundary togenerate an inflated surface that provides a three-dimensional geometriceffect, wherein the three-dimensional geometric effect flows in thespecified flow direction.
 21. The system as recited in claim 13,wherein, to inflate the initial tessellated surface, the surfaceinflation tool is configured to perform the inflation according to alinear system, wherein said linear system does not require thetessellation to extend beyond the boundary.
 22. The system as recited inclaim 13, wherein, to inflate the initial tessellated surface, thesurface inflation tool is configured to perform the inflation by solvinga single linear system.
 23. The system as recited in claim 13, whereinthe surface inflation tool is configured to: specify a positionconstraint on the inflated surface; specify a smoothness value for theposition constraint, wherein the smoothness value indicates a sharpposition constraint or a smooth position constraint; and generate a newinflated surface that provides a different three-dimensional geometriceffect according to the position constraint and the specified smoothnessvalue, wherein the new inflated surface passes through the positionconstraint but does not maintain geometric smoothness across theposition constraint if the smoothness value indicates a sharp positionconstraint, and wherein the new inflated surface passes through theposition constraint while maintaining smoothness across the positionconstraint curve if the smoothness value indicates a smooth positionconstraint.
 24. A computer-readable storage medium storing programinstructions, wherein the program instructions are computer-executableto implement: tessellating a surface bounded by a closed two-dimensionalboundary to generate an initial tessellated surface, wherein the initialtessellated surface is bounded by the boundary; specifying a meancurvature constraint value at one or more boundary vertices of theinitial tessellated surface; inflating the initial tessellated surfaceaccording to the specified mean curvature constraint values at the oneor more boundary vertices while maintaining the boundary to generate aninflated surface that provides a three-dimensional geometric effect; anddisplaying the boundary and the three-dimensional geometric effect as aninflated three-dimensional object.
 25. The computer-readable storagemedium as recited in claim 24, wherein, in said tessellating, theprogram instructions are computer-executable to implement triangulatingthe surface bounded by the closed two-dimensional boundary, and whereinthe initial tessellated surface is a triangle mesh.
 26. Thecomputer-readable storage medium as recited in claim 24, wherein theprogram instructions are computer-executable to implement: specifying aninternal boundary on the inflated surface; and generating a new inflatedsurface that provides a different three-dimensional geometric effectaccording to the internal boundary and the specified mean curvatureconstraint values at the one or more boundary vertices.
 27. Thecomputer-readable storage medium as recited in claim 26, wherein, insaid generating a new inflated surface, the program instructions arecomputer-executable to implement: flattening the initial inflatedsurface; computing the 2D position of the internal boundary;tessellating the surface subject to the internal boundary to generate anew tessellated surface; specifying a mean curvature constraint value atone or more vertices of the new tessellated surface at the internalboundary; and inflating the new tessellated surface according to thespecified mean curvature constraint values at the one or more boundaryvertices and the mean curvature constraint values at the one or morevertices of the new tessellated surface at the internal boundary. 28.The computer-readable storage medium as recited in claim 27, wherein theprogram instructions are further computer-executable to implement movingthe internal boundary to its original 2D position.
 29. Thecomputer-readable storage medium as recited in claim 26, wherein, insaid specifying an internal boundary on the inflated surface, theprogram instructions are computer-executable to implement receiving userinput via a user interface, wherein the user input indicates theinternal boundary on the displayed inflated three-dimensional object.30. The computer-readable storage medium as recited in claim 24,wherein, in said specifying a mean curvature constraint value at one ormore boundary vertices of the initial tessellated surface, the programinstructions are computer-executable to implement receiving user inputindicating the mean curvature constraint values at the one or moreboundary vertices.
 31. The computer-readable storage medium as recitedin claim 24, wherein the program instructions are computer-executable toimplement: specifying a flow direction for the surface bounded by theclosed two-dimensional boundary; and inflating the initial tessellatedsurface according to the specified mean curvature constraint values atthe one or more boundary vertices and the flow direction for the surfacewhile maintaining the boundary to generate an inflated surface thatprovides a three-dimensional geometric effect, wherein thethree-dimensional geometric effect flows in the specified flowdirection.
 32. The computer-readable storage medium as recited in claim24, wherein said inflating the initial tessellated surface is performedaccording to a linear system, wherein said linear system does notrequire the tessellation to extend beyond the boundary.
 33. Thecomputer-readable storage medium as recited in claim 24, wherein saidinflating the initial tessellated surface is performed by solving asingle linear system.
 34. The computer-readable storage medium asrecited in claim 24, wherein the program instructions arecomputer-executable to implement: specifying a position constraint onthe inflated surface; specifying a smoothness value for the positionconstraint, wherein the smoothness value indicates a sharp positionconstraint or a smooth position constraint; and generating a newinflated surface that provides a different three-dimensional geometriceffect according to the position constraint and the specified smoothnessvalue, wherein the new inflated surface passes through the positionconstraint but does not maintain geometric smoothness across theposition constraint if the smoothness value indicates a sharp positionconstraint, and wherein the new inflated surface passes through theposition constraint while maintaining smoothness across the positionconstraint curve if the smoothness value indicates a smooth positionconstraint.
 35. A computer-implemented method, comprising: executinginstructions on a specific apparatus so that binary digital electronicsignals representing a surface bounded by a closed two-dimensionalboundary are tessellated to generate an initial tessellated surface,wherein the initial tessellated surface is bounded by the boundary;executing instructions on the specific apparatus so that binary digitalelectronic signals representing a mean curvature constraint value arespecified at one or more boundary vertices of the initial tessellatedsurface; executing instructions on the specific apparatus so that binarydigital electronic signals representing the initial tessellated surfaceare inflated according to the specified mean curvature constraint valuesat the one or more boundary vertices while maintaining the boundary togenerate an inflated surface that provides a three-dimensional geometriceffect; and storing the boundary and the inflated surface as an inflatedthree-dimensional object in a memory location of the specific apparatus.